Curiosity in Time

We explore the idea of adding the ability to learn time dependent tasks into the Intelligent Adaptive Curiosity (IAC) learning scheme as presented by Oudeyer, et al. [2004, 2006]. We use a three layer architecture (IGE) consisting of a bottom-level IAC brain, an Elman network, and a toplevel IAC brain. For both IAC brains we use the equilibrium Growing Neural Gas (GNG) algorithm to cluster the regions rather than k-means clustering. The bottom-level IAC brain takes as input the robot’s sensors and motors and outputs actions, while the Elman network takes as input the regions of the lower-level IAC brain and attempts to predict the next region. The upper-level IAC brain takes the hidden unit activations of the Elman network and uses these to decide on actions for the robot. Our environment consists of a square where each corridor of the square has a different color scheme. Thus, each corridor is, in itself, predictable by the original IAC architecture, but the corners where the corridors join require time-dependence to predict. Upon running the experiments, we found that using GNG to cluster the regions allows the robot to learn the environment with only 20-25 regions, which is a significant improvement over the approximately 200 that would be formed by k-means clustering. However, while the robot was able to make accurate predictions in the corridors, it could not learn the time-dependence of the environment. We hypothesize that this occurred because the robot’s behavior when controlled by IAC is too unstructured to be used as input to the Elman network. We conclude, therefore, that an Elman network may be helpful in introducing time-dependence to IAC, but only if trained off-line or if the lower-level brain produces more structured actions than those of IAC.